Enhanced Delta-tolling: Traffic Optimization via Policy Gradient Reinforcement Learning
نویسندگان
چکیده
The prospect of widespread deployment of autonomous vehicles invites the reimagining of the multiagent systems protocols that govern traffic flow in our cities. One such possibility is the introduction of micro-tolling for fine-grained traffic flow optimization. In the micro-tolling paradigm, different toll values are assigned to different links within a congestable traffic network. Self-interested agents then select minimal cost routes, where cost is a function of the travel time and tolls paid. A centralized system manager sets toll values with the objective of inducing a user equilibrium that maximizes the total utility over all agents. A recently proposed algorithm for computing such tolls, denoted Delta-tolling, was shown to yield 32% reduction in total travel time in a simulated traffic scenario compared to when there are no tolls. Delta-tolling includes two global parameters: beta which is a proportionality parameter, and R which influences the rate of change of toll values across all links. This talk will introduce a generalization of Delta-tolling which accounts for different beta and R values on each link in the network. While this enhanced Delta-tolling algorithm requires setting significantly more parameters, we show that they can be tuned effectively via policy gradient reinforcement learning. Experimental results from several traffic scenarios indicate that Enhanced Delta-tolling reduces total travel time by up to 33% compared to the original Delta-tolling algorithm, and by up to 52% compared to not tolling.
منابع مشابه
Natural Actor-Critic for Road Traffic Optimisation
Current road-traffic optimisation practice around the world is a combination of hand tuned policies with a small degree of automatic adaption. Even state-ofthe-art research controllers need good models of the road traffic, which cannot be obtained directly from existing sensors. We use a policy-gradient reinforcement learning approach to directly optimise the traffic signals, mapping currently ...
متن کاملTraffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning
Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict t...
متن کاملLearning Road Traffic Control: Towards Practical Traffic Control Using Policy Gradients Diplomarbeit
The optimal control of traffic lights in urban road networks is a highly complex problem. Many factors influence the flow of traffic, and hence the performance of a traffic network, of which few can readily be measured. Currently used control systems are often relatively simple and date back several decades, while more sophisticated optimisation methods fail for large networks. Reinforcement le...
متن کاملStochastic Variance Reduction for Policy Gradient Estimation
Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stochastic variance reduced gradient descent (SVRG) technique [1] to model-free p...
متن کاملDelta-Tolling: Adaptive Tolling for Optimizing Traffic Throughput
In recent years, the automotive industry has been rapidly advancing toward connected vehicles with higher degrees of autonomous capabilities. This trend opens up many new possibilities for AI-based efficient traffic management. This paper investigates traffic optimization through the setting and broadcasting of dynamic and adaptive tolls under the assumption that the cars will be able to contin...
متن کامل